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I. OVERVIEW 

Applied Expertise, Inc. was tasked to provide a constructive appraisal of the Concept Document 
of the Repository-Based Software Engineering Program (December 11, 1991) by the Level 3 
Program Manager. The following is a response to this request. 

Introduction 

The Concept Document is designed to provide an overview of the Repository-Based Software 
Engineering (RBSE) Program. The Document should be brief and provide the context for 
reading subsequent requirements and product specifications. That is, all requirements to be 
developed should be traceable to the Concept Document. 

Applied Expertise’s analysis of the Document was directed toward assuring that: (i) the Executive 
Summary provides a clear, concise, and comprehensive overview of the Concept (rewrite as 
necessary), (ii) the sections of the Document make best use of the NASA "Data Item 
Description" for concept documents, (iii) the information contained in the Document provides 
a foundation for subsequent requirements, and (iv) the document adequately: 

• identifies the problem being addressed, 

• articulates RBSE’s specific role, 

• specifies the unique aspects of the program, and 

• identifies the nature and extent of the program’s users. 

Methodology and Scope 

Our approach to the analysis included: 

• An appraisal of the Document with a view toward evaluating its organization and identifying 
pertinent information that is missing or in need of clarification. (See Exhibit A for 
Organization Template) 

• A section-by-section analysis of the Document to assure that each section tracks with NASA 
guidelines and contains the information considered most relevant to providing the reader with 
a clear understanding of the problem being addressed by the Program and a clear description 
of the program’s purposes, goals, objectives, benefits and limitations. 

• A restructuring of the Executive Summary with a view toward providing a clear, concise and 
comprehensive presentation for the Executive reader. (See Exhibit B for Revised Preliminary 
Executive Summary) 
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The analysis was limited to the Concept Document dated December 11, 1991, and did not 
include discussions with those who prepared the Document or review of all the references and 
other materials used by the Document originators. 

Major Concerns 

The Concept Document follows the broad outline presented in NASA-DID-P100, Concept Data 
Item Description. On the basis of our analysis, however, we believe the Document is too 
fragmented and needs major organizational restructuring within the NASA guidelines. We also 
believe the Document should: 

1 . Specifically define and bound the problem that RBSE seeks to solve and delineate which parts 
of the problem RBSE is addressing. 

2. Clarify RBSE’s functions in terms of (i) operations, (ii) system development, (iii) research 
and how these functions interrelate. Moreover, the document should specify the types of 
products the program will deliver to customers. 

3. Carve out the program’s special niche by identifying the specific customer groups RBSE will 
serve. For example, automotive, manufacturing, and avionics control systems are all 
potential customers for safety-critical software solutions. 

4. Make a clearer distinction between what the repository is currently doing and what more it 
will do when the RBSE program is completed. As presented, the document does not provide 
any illustrations demonstrating that successes achieved under the existing repository provide 
a rationale for continuation and expansion. 

5. Provide evidence or documentation that substantiates or gives credibility to the benefits 
asserted in the Document. We need to more clearly demonstrate that the costs to be incurred 
in completing RBSE will be commensurate with the benefits to be achieved. 

6. Reflect quantified measures of current use and projected use. This would further demonstrate 
the growing importance and reliance on reuse. 

7. Address the issue of self-sufficiency. 

Identification of problem areas, organization restructuring and suggested replacement text are 
described in greater detail in the following Section-by-Section Analysis. 
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n. SECTION-BY-SECTION ANALYSIS 

This section-by-section analysis of the Concept Document is designed to describe the purpose of 
each major section of the Document, provide an appraisal of the organization and information 
included in each section and subsection and, where appropriate, provide examples of suggested 
replacement text. 

Section 1.0 Introduction 

Purpose - This section of the Concept Document is intended to provide the reader with an 
identification of the Document, its purposes, scope, status and organization. 

Appraisal - Our analysis shows that Section 1 .0 of the Concept Document could be further 
improved by presenting the required information in a more organized and descriptive manner. 

Replacement Text - To illustrate what we believe to be a more comprehensive introduction 
section, we have completely revised Section 1.0 for your consideration. (See Exhibit C, 
” Suggested Introduction ") 

Section 2.0 Related Documentation 

Purpose - This section should provide identification of and reference to laws, regulations, 
standards and other informational documents providing background, historical perspective and 
related technical developments. The section’s subsections should reflect applicable and 
informational documents. 

Appraisal - The references included in this section should be reviewed to ensure they are 
current. For example, the current version of the NASA Software Documentation Standard 
Software Engineering Program (SMAP) is not referenced. Also, any additional references used 
to revise the Concept Document should be added to this section. 

Replacement Text - None suggested 

Section 3.0 Definition of the RBSE Program 

Purpose - This section should (i) define the Program, identify the problem to be resolved, and 
provide necessary background information; (ii) articulate long-term goals, specific objectives, 
and describe benefits; (iii) identify the unique aspects of the Program and Program activities; (iv) 
define the approach and implementation of the Program; and describe Program policies. 
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Appraisal - We believe this section needs major revision. It does not distinctly set off the 
problem, or make a clear distinction between the current and proposed RBSE. Background 
information is discussed throughout the section without any clear focus. The section does not 
provide sufficient information about what is unique about RBSE or provide the reader with a 
clear statement of the specific objectives of RBSE. Also, there is no development of tangible 
benefits to be derived from RBSE. In addition, this section does not logically define the 
Program or provide sufficient information on the research, systems development, or 
organizational activities of the Program and their interrelationships. Exhibit A of this analysis 
suggests an overall organizational template for a revised Concept Document. Section 3.0 of the 
template categorizes by subsections the types of information we believe should be included in the 
section. 

The following highlights some specific problems: 

1. The first paragraph under Section 3.0 starts by describing the Concept Document. This was 
done in Section 1.0 Introduction and need not be repeated. See suggested language under 
replacement text. 

2. Subsection 3.1 does not make a clear statement about the purpose and scope of the program. 
See suggested language under replacement text. Also, the scope of the program needs better 
identification. This can be accomplished by abstracting and condensing the bulleted 
information appearing on page 3-1 of the Concept Document. 

3. The bulleted information appearing under subsection 3.1 on page 3-1 is repeated verbatim 
under Section 3-2 Goals and Objectives. This is unnecessary redundancy and appears to be 
presented as both Program scope and objectives. 

4. The current Document does not provide sufficient information about the different elements 
of the problem and those aspects to be addressed by RBSE. We suggest this information be 
developed under a separate side caption 3.1.1 Problem Statement. See suggested language 
for introducing the problem. 

5. Currently, background information is spread throughout the Concept Document. We believe 
it needs to be pulled together in one place. We suggest a background section be developed 
under a separate Subsection 3.1.2 to provide the reader with a brief statement about the role 
of the software industry and an overview of the history and current status of the reuse library. 
This background section should reflect: 

a. Role of the Software Industry - Software plays an expanding and an increasingly critical 
role in the safety, quality and competitiveness of U.S. products and services etc. 
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b. Project History - Members of the RBSE team developed and currently operate a 

prototype repository library etc. 

c. Current Status - The status of the current library etc. provides a foundation for 

the RBSE etc. 

6. Section 3.2 covering goals and objectives needs to provide a clearer definition of the 
objectives of RBSE. The section provides the reader with a picture of the long-term goals 
but does not clearly identify the specific objectives. As presented, the long-term goals would 
be difficult to measure and attribute to RBSE, whereas specific objectives would be more 
measurable. We believe the Document could be improved by identifying and describing the 
specific goals of RBSE. We suggest it be reorganized by developing a number of subsections 
along the following lines: 

a. 3.2.1 Long-Term Goals - The information relating to the improvement of U.S. 
economic growth, productivity, and competitiveness should be discussed in this 
subsection. Specifically, the goal of improving software practices among RBSE’s target 
customers should be discussed here. 

b. 3.2.2 Specific Objectives - The specific objectives of RBSE should be defined and 
described in this section. 

c. 3.2.3 Program Benefits and Measurements - This subsection should describe the benefits 
of RBSE. To the extent practical these benefits should be presented in terms of dollar 
savings. There are a number of studies that contain data relating to savings that can be 
attributed to reuse. One such example is NASA Software Engineering Laboratory’s 
"Experiments in Software Engineering Technology: Recent Studies in the SEL (1990- 
1991)." In addition, the Document should describe the process to be used to measure 
the benefits derived from achieving Program results. 

7. We believe Section 3.3 Program Description should be reorganized to provide those 
reviewing and approving the Document with a clearer picture of RBSE and what the Program 
is to accomplish. Also, this section should describe the unique role of RBSE. To accomplish 
this, we suggest the Program Description section be segregated into a number of subsections, 
as follows: 

a. 3.3.1 The Unique Role of RBSE - This subsection should describe the overall solution 
to software engineering problems and the unique role RBSE will play in contributing to 
the solution of this problem. Those aspects of the problem being addressed by RBSE 
should be highlighted. We suggest this can be done by developing a graphic. For 
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example, graphics might be used to highlight the different aspects of the problem and 
pinpoint the areas RBSE will address (while illustrating roles played by other repository 
programs). 

b. 3.3.2 Research Activities - This subsection should be developed to describe the research 
activities of the RBSE and discuss their benefits to RBSE and its goals. 

c. 3.3.3 Systems Development - This subsection should identify and describe the 
improvements to the repository to be developed under the Program. 

d. 3.3.4 Operation - This subsection should describe RBSE operation. For example, an 
expansion of the information described in 6.4 Personnel would fit into this section. 

e. 3.3.5 Interoperability - This subsection should discuss interoperability as a method for 
cost-effectively broadening RBSE’s customer and supplier base. It should specifically 
describe the Reuse Library Interoperability Group (RIG) and outline RBSE’s activities 
with respect to interoperability. 

f. 3.3.6 Total Quality Management (TQM) - This subsection should discuss TQM as a 
cornerstone of the Program’s focus, impact, value to its customers and efficiency. It 
should affirm the Program’s intent to refine and achieve specific, measurable goals. 

g. 3.3.7 Cost Recovery - This subsection should address the issue of cost recovery. 

In summary, we are suggesting that Section 3.0 be reorganized and that additional information 
as discussed above be included in the section. We suggest the reorganization follow the structure 
detailed in Exhibit A, "Organization Template." 

Replacement Text - The following represents examples of the replacement text for selected areas 
discussed above. 

3.0 DEFINITION OF THE REPOSITORY-BASED SOFTWARE ENGINEERING (RBSE) 
PROGRAM (replacement text) 

RBSE is a NASA program that encompasses a public-domain life-cycle repository designed to 
collect, store, and distribute a unique set of reusable products, processes, interfaces, and 
information spanning the software engineering life cycle. RBSE will also maintain standards, 
practices and guidelines relating to life-cycle repositories and to repository-based approaches to 
software engineering. Through targeted research, the Program fills in critical technology gaps. 
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3.1 Purpose and Scope 

Purpose - RBSE is a research and development program designed to contribute to the mainstream 
adoption of software reuse in government, industry, and academia. Its broad purpose is to 
provide a repository that nourishes the selection, adaptation, and reuse of existing components 
and that promotes common practices and standards. 

3.1.1 Problem Statement 

Current software development practices need the clarity, consistency, and predictability that 
other, more mature engineering disciplines provide as a matter of course. For example, 
buildings, bridges and even computer hardware are built by using common models, practices, 
guidelines, interfaces, and off-the-shelf parts. These elements, taken for granted in other 
engineering disciplines, are in their infancy for software engineering practice. a result, 
opportunities to improve the quality, safety, and rapid tailorability of software is lost. Moreover, 
costs are increased and the competitiveness ofU.S. products and services are diminished. 

Section 4.0 User Definition 

Purpose - This section identifies specific users of RBSE, the functional capabilities and products 
required by those users, and the unique niche established by the program for specific types of 
users. 

Appraisal - This section needs to be more specific concerning who the users are and how they 
will be using the system. The descriptions used in this section should correspond with the 
scenarios presented in Section 6.0 Operational Scenarios. The section could be made much 
clearer if segregated into different sections as follows: 

a. 4. 1 User Category - See suggested replacement text below. 

b. 4.2 Target Customers - This subsection could be used to elaborate on the special 
relationship RBSE intends to develop and maintain with target industries, particularly 
those heavily involved in safety-critical systems, and commercial aerospace. This 
subsection should also address the demands of these target customers. 

c. 4.3 Customer Use of the Repository - This section should incorporate the information 
reflected in subsection 6.5 captioned equipment and such other information describing 
how RBSE customers will use the system. Section 6.5 describes how users access the 
system and should be deleted from Section 6.0. 
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Replacement Text - The following represents examples of replacement text: 

4. 1 User Categories - The KBSE library will be available to three broad categories of users. 
They are submitters, reusers, and repository staff. 

• Submitters include government and commercial software developers, authors, conference 
organizers, and marketers. The submitter would provide the library a set of artifacts for 
qualification and admission into the life-cycle repository. ( See Scenario 6.1 - Submission to 
the Repository) 

• Reusers include software engineers, system architects, managers, educators, authors, and 
conference organizers. The reuser locates and retrieves components that can be used and/or 
adapted for use in the development of a new software engineering system. (See Scenario 6.2- 
Reuse of Repository Con tents) 

• Repository staff includes the RBSE operations and support staff. Staff will acquire, classify, 
qualify, and maintain the repository contents and organization. (See Scenario 6.3 - 
Repository Maintenance) 

Section 5.0 Capabilities and Characteristics 

Purpose - This section presents the general capabilities and resources of RBSE and projects the 
expected usage of the Life-Cycle Repository by the different classes of users. 

Appraisal - This section should include and expand upon the comments made in Section 4.0 with 
regard to the unique niche to be addressed by RBSE. 

Replacement Text - None Suggested 

Section 6.0 Operational Scenarios 

Purpose - This section presents a number of scenarios illustrating user interaction with the 
repository. 

Appraisal - This section should be modified by moving Section 6.4 and including the 
information under Subsection 3.3.3 Operations. Also, Section 6.5 should be moved and included 
under proposed section 4.3 Customer Use of the Repository. 

Replacement Text - None Suggested 
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Section 7.0 Abbreviations. Acronyms and Definitions 

Purpose - This section contains an alphabetized list of definitions for abbreviations, acronyms, 
and special or unusual terms used in the Document. 

Appraisal - The Document does not include a Section 7.0 but does have a page dealing with 
acronyms. We believe a section should be added that covers abbreviations, acronyms, and 
definitions. For example, reuse as discussed within the framework of RBSE may be somewhat 
narrower than reuse discussed by others — that is, off-the-shelf commercially available 
components. 

Replacement Text - None Suggested 
A ppendices 

Purpose - To provide information relating to the Program being discussed in the Concept 
Document. 

Appraisal - Appendix information should be informative and relevant but not essential to an 
understanding of the Concept Document. Any essential information included in the appendices 
should be extracted and included where appropriate in the body of the Document. 

Replacement Text - None suggested 
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EXHIBIT A - ORGANIZATION TEMPLATE 


1 INTRODUCTION 

1.1 Identification 

1.2 Scope 

1.3 Purpose and Objectives 

1.4 Status 

1.5 Organization 

2 Related Documentation 

2. 1 Applicable Documents 

2.2 Informational Documents 

3 Definition of the Repository-Based Software Engineering (RBSE) Program 

3.1 Purpose and Scope 

3.1.1 Problem Statement 

3.1.2 Background 

3.2 Goals and Objectives 

3.2.1 Long-Term Goals 

3.2.2 Specific Objectives 

3.2.3 Program Benefits and Measurements 

3.3 Program Description 

3.3. 1 The Unique Role of RBSE 

3.3.2 Research Activities 

3.3.3 Systems Development 

3.3.4 Operations 

3.3.5 Interoperability 

3.3.6 Total Quality Management 

3.3.7 Cost Recovery 

3.4 Policies 

4 User Definition 

4. 1 User Categories 

4.2 Target Customers 

4.3 Customer Use of Repository 
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5 Capabilities and Characteristics 

5.1 Program Capabilities 

5.2 Characteristics 

6 Operational Scenarios 

6.1 Scenario #1 - Repository Submissions 

6.2 Scenario #2 - Reuse of Repository Contents 

6.3 Scenario #3 - Repository Maintenance 

7 Definitions, Acronyms, and References 

7.1 Definitions 

7.2 Acronyms 

7.3 References 
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EXHIBIT B - SUGGESTED EXECUTIVE SUMMARY (preliminary) 


THE PROBLEM 

Software plays an expanding and increasingly critical role in the safety, quality and 
competitiveness of U.S. products and services. Despite its importance, software 
engineering and development practices lack the clarity, consistency, and predictability 
provided by more mature engineering disciplines. For example, engineering practices for 
erecting, constructing, and manufacturing buildings, bridges and computer hardware 
include the reuse of common models, guidelines, interfaces, and off-the-shelf components. 
These elements, taken for granted in other engineering disciplines, are not consistently 
applied in software development. Today’s software is commonly built haphazardly. All 
too often, efforts to improve the situation through reusing what exists, or engineering 
products for later reuse, are dropped to meet schedule or budget pressure. 

As a result, a disturbingly high proportion of software continues to be brittle, inflexible 
and hard to maintain more than most of the time. Opportunities are lost to improve the 
quality, safety, and rapid tailorability of software. Perhaps more importantly, costs and 
risks to the taxpayer, consumer and shareholder are inflated and the competitiveness of 
U.S. products and services is diminished. 

DEFINITION, GOALS AND OBJECTIVES 

NASA’s Repository-Based Software Engineering (RBSE) program comprises three areas: 
(i) operation of a national software life-cycle repository to promote software reuse, (ii) 
system development to support this operation and (iii) research to increase Program 
impact. The goal of this program is to improve software practices among RBSE’s target 
customers -- software developers in key segments of U.S. industry — so their software 
engineering efforts parallel the clarity, consistency and predictability of other engineering 
disciplines. As a result, these customers will be able to develop better and safer software 
that is produced faster and at lower cost. 

Software reuse has been recognized by software practitioners as well as policy makers as 
a key factor in improving product quality and competitiveness. There is growing evidence 
that products that are engineered for reuse are more reliable, more maintainable and of 
higher quality than conventional software products. RBSE, through its strategy of 
cooperative participation in national programs designed to encourage the mainstream 
adoption of software reuse, will make a significant contribution toward this goal by — 
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• delivering and supporting a robust set of reuse products geared to help customers 
incorporate effective reuse into the way their organizations do business. (Section 
3.3.2) 

• improving the effectiveness of the repository (the product delivery mechanism) 
through application of research results, customer feedback and off-the-shelf software. 
(Section 3.3.3) 

• filling in critical technology gaps through research, such as methods for organizing 
and managing software artifacts and process models. (Section 3.3.4) 

• broadening the customer and supplier base by supporting interoperability. (Section 
3.3.5) 

RBSE was the first program to evolve the concept of life-cycle repository-based 
engineering, which is now being implemented by a number of organizations in 
government and industry. This multiplicity, when balanced with cooperation, is critical 
while reuse matures in technology and practice. It allows each of the individual reuse 
libraries to customize its technology and services to the needs of its customers. Each 
repository, in developing creative approaches to practical problems, can benefit from the 
advances and inventiveness of other repositories. To this end, the Reuse Library 
Interoperability Group (RIG) develops consensual standards for the interoperability of 
software reuse libraries. RBSE participates in the RIG efforts to make interoperability a 
reality. 

No matter how sophisticated the repository becomes, qualified repository staff who are 
willing and able to help their customers will mean the difference between mainstream and 
casual use by major organizations. Similarly, a program-wide commitment to quality — 
the measured process of continuously providing customers with what they expect and need 
most, ever more efficiently and effectively — is key to success in any and all Program 
objectives and goals. (Section 3.3.6) 

USERS 

In order to get the greatest leverage from its efforts, RBSE seeks to develop a loyal 
customer base within two overlapping customer groups. Those groups are (i) builders of 
software-intensive, safety-critical systems, including manufacturing, railroads, medical, 
nuclear and hazardous material, and (ii) NASA and civilian aerospace. RBSE is uniquely 
positioned to become the supplier of choice for NASA-developed software artifacts and 
related documents. 
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IMPACT 

There are three distinguishing features of the RBSE program. It is committed (i) to 
becoming the supplier of choice for reusable artifacts from civilian aerospace application 
domains, (ii) to improving software-intensive, safety-critical systems and (iii) to 
introducing reuse into all phases of the software life cycle. 

Industry studies indicate that five- to ten-year productivity gains of over 25 percent, 
reliability improvements of 300 percent and 60 percent reductions in the time it takes to 
deliver a product are achievable through software engineering and reuse. RBSE has 
targeted key, but limited, segments of U.S. industry where the U.S. maintains a 
competitive edge and from which technological successes are likely to spread quickly. 
RBSE, in concert with other reuse and software engineering initiatives, can effect broad 
and positive changes in the way U.S. industry develops software. 

In sum, software theory and practice lack several essential elements common to more 
mature fields of engineering. By building a strong repository-based program, RBSE will 
promote the reuse of common models, standards, practices, guidelines and off-the-shelf 
components. The success of RBSE and related programs can transform software 
development and support from a cottage industry to an efficient, effective, quality-driven 
industrial process fostering needed improvements in U.S. productivity, economic growth, 
and competitiveness. 
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EXHIBIT C - SUGGESTED INTRODUCTION 


1.0 INTRODUCTION 

1.1 Identification - This is the Concept Document for a Repository-Based Software 
Engineering (RBSE) Program. The Document provides an overview of RBSE and creates the 
foundation for establishing the Requirements Section of the Program’s specifications. 

1 .2 Scope - The Document (i) identifies problems to be addressed and describes the mission, 
goals, and objectives of RBSE; (ii) discusses the relationship and coordination with other related 
activities, (iii) details the general approach toward resolving the problems identified; and (iv) 
identifies intended users and other beneficiaries. 

1.3 Purpose and Objectives - The Document provides the foundation for developing RBSE 
user and system requirements. The Program Management Plan to be developed from the 
Document will discuss how resources are to be organized and applied to achieve the mission, 
goals, and objectives of RBSE. 

1 .4 Status - It is an active Document subject to review and updating to identify problems, 
implement improvements and reflect lessons learned. The scope of the Document will be 
periodically changed as appropriate and necessary. 

1.5 Organization - The Document is comprised of six sections including this introduction 
section. The other sections are: 

• 2.0 Related Documentation - This section provides identification of and reference to 
laws, regulations, standards and other informational documents providing background, 
historical perspective and related technical developments. 

• 3.0 Definition of the RBSE Program - This section provides Program background; 
establishes motivation and problem identification; identifies products, services, and 
benefits; and defines the approach and implementation of the Program. 

• 4.0 User Definition - This section identifies specific users of RBSE, the functional 
capabilities and products required by those users, and the unique niche established by 
the program for specific types of users. 
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• 5.0 Capabilities and Characteristics - This section presents the general capabilities and 
resources of RBSE and projects the expected usage of the Life Cycle Repository by 
the different classes of users. 

• 6.0 Operational Scenarios - This section presents a number of scenarios illustrating 
user interaction with the repository. 

• 7.0 Abbreviations. Acronyms and Glossary - This section contains an alphabetized 
list of definitions for abbreviations, acronyms, and special or unusual terms used in 
the Document. 
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